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Abstract 

This paper is devoted to tiie finite-lengtli analysis of turbo decoding over the binary erasure channel (BEC). 
The performance of iterative belief-propagation (BP) decoding of low-density parity-check (LDPC) codes over the 
BEC can be characterized in terms of stopping sets. We describe turbo decoding on the BEC which is simpler 
than turbo decoding on other channels. We then adapt the concept of stopping sets to turbo decoding and state an 
exact condition for decoding failure. Apply turbo decoding until the transmitted codeword has been recovered, or the 
decoder fails to progress further. Then the set of erased positions that will remain when the decoder stops is equal 
to the unique maximum-size turbo stopping set which is also a subset of the set of erased positions. Furthermore, 
we present some improvements of the basic turbo decoding algorithm on the BEC. The proposed improved turbo 
decoding algorithm has substantially better error performance as illustrated by the given simulation results. Finally, 
we give an expression for the turbo stopping set size enumerating function under the uniform interleaver assumption, 
and an efficient enumeration algorithm of small-size turbo stopping sets for a particular interleaver. The solution is 
based on the algorithm proposed by Garello et al. in 2001 to compute an exhaustive list of all low-weight codewords 
in a turbo code. 

Index Terms 

Binary erasure channel, improved decoding, stopping set, turbo decoding, uniform interleaver, weight spectrum. 

I. Introduction 

Low-density parity-check (LDPC) codes as opposed to turbo codes have been studied extensively on the binary 
erasure channel (BEC). In [1], an iterative decoding algorithm for LDPC codes over the BEC was proposed, and it 
was shown that this scheme approaches channel capacity arbitrarily close. Although carefully optimized irregular 
LDPC codes with iterative decoding can achieve channel capacity on the BEC as the code length tends to infinity, 
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there is still some performance loss compared to maximum-likelihood (ML) decoding of a given fixed code of finite 
length. Recently, some progress has been made towards efficient ML or near ML decoding of LDPC codes over 
the BEC [2]-[4]. The Tanner graph representation of an LDPC code is a bipartite graph with left and right nodes. 
The left nodes correspond to codeword bits. The right nodes correspond to parity-check constraints. It is known 
that iterative belief-propagation (BP) decoding fails if and only if the set of erased bit-positions contains a stopping 
set [5]. A stopping set is a subset of the bit-positions such that the corresponding left nodes in the Tanner graph 
have the property that all neighboring nodes are connected to the set at least twice. 

In this work we consider turbo decoding over the BEC. Turbo codes have gained considerable attention since their 
introduction by Berrou et al. [6] in 1993 due to their near-capacity performance and low decoding complexity. Here 
we consider the conventional turbo code which is the parallel concatenation of two identical recursive systematic 
convolutional encoders separated by a pseudo-random interleaver To accurately describe turbo decoding on the 
BEC we introduce the concept of a turbo stopping set, and we identify an exact condition for decoding failure. 
Assume that we transmit codewords of a turbo code over the BEC. Apply turbo decoding until either the codeword 
has been recovered, or the decoder fails to progress further Then the set of erased positions that will remain when 
the decoder stops is equal to the unique maximum-size turbo stopping set which is also a subset of the set of 
erased positions. We also consider improved turbo decoding on the BEC. The algorithm applies turbo decoding 
until the transmitted codeword is recovered, or the decoder fails to progress further. Then, an unknown (systematic) 
bit-position is identified and its value is guessed, after which turbo decoding is applied again. Thus, the algorithm 
is based on guessing bit-values in erased bit-positions when turbo decoding does not progress further and has the 
same structure as the algorithms in [4]. 

Recently, several algorithms have been introduced to compute the first few terms of the weight distribution of 
both parallel and serial turbo codes. Both exact algorithms (e.g., [7]-[9]) and approximate algorithms (e.g., [10]- 
[12]) have been presented. In this work we also show that basically all trellis-based algorithms can be adapted to 
find the first few terms of the turbo stopping set size enumerating function. In particular, we have considered in 
detail how to adapt the algorithm by Garello et al. introduced in [8] and the improved algorithm in [7]. Also, an 
expression for the (average) turbo stopping set size enumerating function under the uniform interleaver assumption 
is presented. 

Using linear programming (LP) to decode binary linear codes has recently been considered by Feldman et al. 
[13]. See also the seminal papers [14], [15] where LP decoding of turbo-like codes is considered. In particular, 
repeat-accumulate (RA) codes are considered. A description of LP decoding of arbitrary concatenated codes is 
given in [16, Ch. 6]. The obvious polytope for LP decoding is the convex hull of all codewords, in which case LP 
decoding is equivalent to ML decoding. However, the convex hull has a description complexity that is exponential 
in the codeword length for a general binary linear code. Thus, Feldman et al. [13] proposed a relaxed polytope 
which contains all valid codewords as vertices, but also additional non-codeword vertices. The vertices of the 
relaxed polytope are basically what the authors called pseudo-codewords in [13]. One desirable property of the LP 
decoder is the ML certificate property, i.e., when the LP decoder outputs a codeword, it is guaranteed to be the ML 
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codeword. Experimental results with LDPC codes show that the performance of the relaxed LP decoder is better 
than with the iterative min-sum algorithm, but slightly worse than with iterative BP decoding. 

Recently, some understanding of the performance of iterative BP decoding of finite-length LDPC codes over 
general memoryless channels have been developed. Finite graph covers of the Tanner graph and the codes defined 
by these covers play an essential role in the analysis [17]. The low complexity of iterative BP decoding is due to 
the fact that the algorithm operates locally on the Tanner graph of the code. This property is also the main weakness 
of iterative BP decoding, since the decoder cannot distinguish if it operates on the original Tanner graph or on any 
of the finite covers. Hence, codewords in the code defined by a finite cover of the Tanner graph will influence on 
iterative decoding. These codewords are basically what Vontobel and Koetter referred to as pseudo-codewords in 
[17]. It turns out that the set of pseudo-codewords of all finite covers of the Tanner graph is equal to the set of points 
where all entries are rational numbers from the relaxed polytope of LP decoding of LDPC codes as introduced 
by Feldman et al. in [13]. A similar connection between the relaxed polytope of LP decoding of turbo codes, as 
described in [16, Ch. 6], and the pseudo-codewords of all finite covers of the turbo code factor graph [18] was 
established in [19]. Furthermore, Rosnes also showed in [19] that there is a many-to-one correspondence between 
these pseudo-codewords and turbo stopping sets in the following sense. The support set of any pseudo-codeword, 
i.e., the set of non-zero coordinates, is a turbo stopping set, and for any turbo stopping set there is a pseudo- 
codeword with support set equal to the turbo stopping set. A similar connection also holds for pseudo-codewords 
of finite graph covers of Tanner graphs and stopping sets [17], [20]. 

For LDPC codes it has been observed that stopping sets, to some degree, also reflect the performance of iterative 
decoding for other channels than the BEC [17], [21]. It is therefore our hope that the notion of turbo stopping sets 
also can provide some useful insight into turbo decoding on the additive white Gaussian noise (AWGN) channel 
or on other memoryless channels. In [19], Rosnes presented some simulation results to indicate that this may be 
the case. 

This paper is organized as follows. In Section |ll] we define some basic notation and describe simplified turbo 
decoding on the BEC. Section |lll| introduces the concept of a turbo stopping set. We further give some of the basic 
properties and show that turbo stopping sets characterize exactly the performance of turbo decoding on the BEC. 
An improved turbo decoding algorithm on the BEC is introduced in Section Hvl and its superiority compared to 
conventional turbo decoding is illustrated by simulation examples. Finally, in Section|V] we consider enumeration of 
small-size turbo stopping sets for a particular interleaver and under the uniform interleaver assumption. Conclusions 
and a discussion of future work are given in Section IVTI 

II. Preliminaries 

In this section we introduce the channel, define some basic notation, and describe simplified turbo decoding on 
the BEC. 
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A. The BEC 

The BEC model was introduced by Elias [22] in 1955. The channel has recently been used for modeling 
information transmission over the Internet. The BEC is a two-input, three-output discrete memoryless channel. 
Each input bit is erased with probability e, or received correctly with probability 1 — e. 

B. Some Definitions and Basic Notation 

Let C = C{K,Ca,Cb,TT) denote a parallel concatenated convolutional code (PCCC), or turbo code, with infor- 
mation length K, constituent encoders Ca and Cb of rate R ~ k/n, and interleaver tt. In this work we consider 
dual termination [23], which implies that the length of the interleaver I ~ K + 2iy where u is the constraint length. 
We assume here that / is a multiple of k. Let Na and Ni, denote the lengths of the constituent codes. The length 
of the turbo code is denoted by N. For an unpunctured turbo code, Na = N^ = I/R and N = {2/R — 1)1. In 
general, the values of Na, N^, and N depend on the puncturing pattern P and the termination scheme. 

We will now define some useful mappings, but we advise the reader that the formal definitions below will be 
easier to understand after taking a look at Fig. 

Define two mappings fia ■ Zj^ Z^^ U {*} and /i^ : Zfq Z^^ U {*} where Zm ~ {0, 1, . . . , iV — 1} for a 
positive integer N . The mapping /1q gives the index in the first constituent codeword of the turbo codeword index if 
such a relation exists, or * if not. Similarly, the mapping /if, gives the index in the second constituent codeword of 
the turbo codeword index if such a relation exists, or * if not. Note that for turbo codeword indices that correspond 
to systematic bits, the interleaver is used to get the correct constituent codeword index for the second encoder. 

Define two mappings ipa '■ Z^^ Zi U {*} and V'h : Zf4^ Zj (J {*}. The mapping gives the systematic 
sequence index of the first constituent codeword index if such a relation exists, or * if not. Similarly, the mapping 
ipb gives the interleaved systematic sequence index of the second constituent codeword index if such a relation 
exists, or * if not. 

Example 1: Consider a turbo code composed of two identical nominal rate-1/2 constituent convolutional codes. 
The interleaver length 1 — 6 and parity bits from the two constituent encoders are punctured alternatively to create a 
nominal rate-1/2 turbo code. The lengths of the first and second constituent codes are Na = Nb = 9. The interleaver 
TT is defined by {3, 5, 1, 4, 0, 2} (i.e., ^ 3, 1 — > 5, and so on). The ordering of bits in the turbo codeword is 

where li and P°- denote the ith systematic and parity bit from the first constituent code, respectively, and where 
Pj' denotes the ith parity bit from the second constituent code. The mappings fia, f^b, ipa, and ipb are depicted 
graphically in Fig. Q] 

C. Turbo Decoding on the BEC 

The aim of turbo decoding on the BEC is to find a set of paths through each constituent code trellis that 
is consistent with the received sequence. The decoding starts with a set of all paths and iteratively eUminates 
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Fig. 1. Graphical representation of the mappings fia, Mb, ^a, and for the turbo code from Example Q] 



those that are inconsistent. This iterative process continues until either there is only one possible path left in each 
constituent trellis (successful decoding), or there is no change from one iteration step to the next. We will describe 
the basic algorithm, but remark that the BEC version of turbo decoding allows more efficient implementations. For 
simplicity, we omit the details. 

Let T^j^fQ denote an information bit-oriented trellis [24]-[26] for constituent code Cx, x ~ a,b. The trellis 
is time-variant, but periodic, and has two edges out of each vertex. For details, see Section IV-A.ll The number 
of trellis depths in 7^;^^^^ is 7+1, i.e., there is one trellis section for each information bit. Below, x denotes the 
complement of x, a; = a, b. 

Like turbo decoding for an AWGN channel, the constituent decoders work with a forward and a backward pass 
through the constituent code trellis T^f^i^, and during these passes the state metric ax \v,j) (resp. (3i^\v,j)) for 
vertex v at trellis depth j, at the ith iteration, is updated. For the BEC, however, the state metrics are boolean 
and define forward (resp. backward) paths that are consistent with the current estimate of the transmitted sequence 
c = (co, . . . , cjv-i)- The decoding for constituent code Cx is performed as follows. 

Initially, we set ai^'(0,0) = pi^\o,I) = f3i"\Q,I) = true; f3^x\v,j) = true for all vertices v at trellis depth 
j in T^nfo, j = 1, ...,/ — 1; and ai^"* (u, 0) = /si^^ (v, I) = (w, /) = false for every non-zero vertex v at trellis 
depths and /, respectively, in T^jJ^. Finally, set the estimate of the transmitted sequence c equal to the received 
sequence. 

Then the forward pass calculates, for j = 1, . . . ,1 and for each vertex v' at trellis depth j in Tj^lf^, Q{v') and 
subsequently ai'\v',j) = {(3^J~'^\v' , j) AND Cj («'))■ Here Qiv') is a boolean variable which is true if there 
exists, in the {j — l)th trellis section of TTjJfo, an edge consistent with c, with right vertex v', and left vertex v such 
that ax\v,j — 1) = true. Initially, prior to the forward pass, we set ax\v,0) = a^x\v,0) for all vertices v at 
trellis depth in T^f^. 

In a similar way the backward pass calculates /Si'-* {v, j) for all vertices v at trellis depth j in 7^,^^, j = 0, . . . , /—I. 
Finally, the estimate c of the transmitted sequence is updated so that only information values consistent with legal 
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edges remain, and control is passed to the other constituent decoder 

The codeword is said to be recovered if Cj ^ *, where ★ denotes an erasure, for all j such that /^ta(j) 7^ * and 
i^aif^aU)) ^ *, j ~ 0, ■ ■ ■ , N — 1. The decoder is said to fail to progress further if, for x ~ a,b, the state metrics 
ax\v,j) and (3x\v,j), for some positive integer / > 1, are equal to the state metrics ai ^\v,j) and I3x ^\v,j), 
respectively, for all vertices v at trellis depth j in T-j^^^, j = 1, . . . , I — 1. 

III. Turbo Stopping Sets 

In this section we will introduce the concept of a turbo stopping set. A turbo stopping set is the equivalent of 
an LDPC stopping set when turbo decoding and not iterative BP decoding is performed. 

Definition 1: Let C denote a given PCCC with interleaver tt. A set 5 = ^(Tr) C {0, . . . , — 1} is a turbo 
stopping set if and only if there exist two linear subcodes Ca C Ca C {0,1}^" and C Cf, C {0,1}^'' of 
dimension > with support sets x(C'a) and x(C'fc), respectively, such that 

x(a) = Ma(5)\{*} 

x(a)=A*b(5)\{*} 

AMxiCa)) \ {*}) = MxiCb)) \ {*}. (1) 

The size of a turbo stopping set S is its cardinality. 

The lemmas below state some of the properties of a turbo stopping set. 

Lemma 1: Let C denote a given PCCC with interleaver tt. The support set of any non-zero codeword from C is a 
turbo stopping set of size equal to the Hamming weight of the given codeword. Thus, the minimum turbo stopping 
set size is upper-bounded by the minimum Hamming weight. 

Proof: Denote the turbo codeword by c and the corresponding first and second constituent codewords by 
Ca and Cf), respectively. Furthermore, let Ca = {ca,ONa} and Cb = {cb,OjV(,} where Ojv^ denotes an all-zero 
sequence of length N^, x — a,b. The result follows immediately from Definition^ since x(C'a) — Ma(x(c)) \ {*}, 
xiCb) = /^6(x(c)) \ M, and ^(Va(Ma(x(c)) \ {*}) \ {*}) - V';,(a^;,(x(c)) \ {*}) \ {*}■ ■ 

Lemma 2: Let C denote a given PCCC with interleaver tt, and let 5 = iS (tt) denote a turbo stopping set. If Ca 
and Cb can both be decomposed into direct sums of linear subcodes of dimension 1 with disjoint support sets, then 
S is the support set of a turbo codeword of Hamming weight |iS|. The converse is also true. 

Proof: Assume that Ca and Cb can both be decomposed into direct sums of linear subcodes of dimension 1 
with disjoint support sets. In more detail, 

Ca = WaKo^A + ■■■ + {ci^), OjvJ and Cb = {c^°\OnJ + ■■■ + {c^'KOnJ 

where p and q are non-negative integers. The codewords Ca = X^iLo '-a and Cb = J2i=o •^i*'' have support sets 
x(ca) = x(C'a) and x(c&) = xi^b), respectively, since the support sets of the direct sum subcodes are disjoint. 
Since the two sets 7r(V'a(yUa(5) \ {*}) \ {*}) = Tr{Mx{ca)) \ {*}) and i'b{pb{S) \ {*}) \ {*} = Mx{cb)) \ {*} 
are equal (from Definition [Q, there exists a turbo codeword c with first and second constituent codewords Ca and 
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c;,, respectively. Thus, the turbo stopping set is the support set of a turbo codeword of Hamming weight |iS|, since 
X(c) = S. 

We prove the converse using the following argument. The turbo stopping set S is the support set of a turbo 
codeword c. The corresponding first and second constituent codewords are denoted by Ca and c;,, respectively. 
Then, there exist subcodes Ca = {ca,ONa} C Ca and Cb = {ch,OAr^} C Cb both of dimension 1 which satisfy 
the constraints in Q with S = x(c), and the result foUows. (The codewords and Cb may or may not be further 
decomposed.) ■ 

Construction 1: Let = {Va, Ea) and Tf, = (Vf,, Eb) denote two arbitrarily chosen Tanner graphs for the first 
and second constituent codes, respectively. The node set Vx can be partitioned into three disjoint subsets 1/f, V]!, 
and V^, corresponding to systematic bits, parity bits, and parity-check equations, respectively, where x = a,b. 
There is an edge e ^ E^ connecting a node & = U Kl' to a node G if and only if the first (x — a) 
or second (x ~ b) constituent codeword bit represented by w!^ is checked by the parity-check equation represented 
by v^- Next, define 

E'b = {{vl,,v^,) : v^, G «D &EbJ = 0,...,I-l}U {«, v^b) : < G V,^ , vl G {vl v^) G Eb}. 

Then, the graph T = (K U U V{;, Ea U E'^^) is a Tanner graph for the turbo code C. 

We remark that for a given binary linear code there exist in general several full-rank parity-check matrices, and 
thus several (distinct) Tanner graphs. The minimum stopping set size is in general a function of the Tanner graph 
[27]-[29]. Note that Construction Ogives a specific class of Tanner graphs for a turbo code that is a proper subset 
of the class of all Tanner graphs of the given turbo code. We will consider this specific class of Tanner graphs 
below. 

Lemma 3: Let C denote a given PCCC with interleaver tt. For any turbo stopping set S = S{tt) there is an LDPC 
stopping set of cardinality |5| in any Tanner graph T for C within the class of Tanner graphs from Construction 

Proof: We use the notation introduced in Construction [2 The set {u^ j : j G fJ,x{S) \ {*}}, where v'!^ ^ 
denotes the jth node in V^, is an LDPC stopping set in any Tanner graph of Cx, since ij,x{S) \ {*} is the 
support set x{C^x) (from Definition Q of some subcode Cx of Cx of dimension > 0. We have here used the fact 
that the variables nodes corresponding to the support set of a non-zero codeword constitute an LDPC stopping set. 
Furthermore, the set 

{{vl^ : J G \ {*}} U : J G fib{S) \ {*}}) \ (2) 

is an LDPC stopping set in the Tanner graph T of the turbo code C due to the last condition in Definition [0 The 
cardinality of the LDPC stopping set in (|2j is |iS|. ■ 
Note that the converse is not necessarily true (i.e., for an LDPC stopping set in any Tanner graph T for C within 
the class of Tanner graphs from Construction there is not necessarily a turbo stopping set). Thus, iterative BP 
decoding using a Tanner graph within the class of Tanner graphs of a turbo code from Construction \l\ is inferior 
to turbo decoding on the BEC. The following theorem states an exact condition for decoding failure. 
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Theorem 1: Let C denote a given PCCC with interleaver tt that we use to transmit information over the BEC. 
The received vectors are decoded using turbo decoding until either the codeword has been recovered, or the decoder 
fails to progress further Then the set of erased positions that will remain when the decoder stops is equal to the 
unique maximum-size turbo stopping set which is also a subset of £, where £ denotes the subset of erased positions. 

Proof: The proof given here is inspired by the proof given by Di et al. in [5, Lemma LI] in the context of 
an LDPC stopping set. Let 5 be a turbo stopping set contained in £. The claim is that the basic turbo decoder 
cannot determine the bits corresponding to the positions in the turbo stopping set S. Assume that all other bits 
are known. Turbo decoding starts by activating the first constituent decoder For the first constituent decoder, the 
forward-backward algorithm will determine \Ca\ > 1 possible paths through the trellis. The support set of these 
possible paths is equal to iJ.a{S) \ {*}, and they are all equally likely. Consequently, no additional codeword bits can 
be determined. Thus, the extrinsic probability distributions for systematic bits in positions in ipa{lJ-a{S) \ {*})\{*} 
are uniform. For the second constituent decoder, the forward-backward algorithm will determine \Ci,\ > 1 possible 
paths through the trellis. The support set of these possible paths is equal to /ih(5) \ {*} and, since the a priori 
probability distributions for systematic bits in positions in TT{tl'a{iji.a{S) \ {*}) \ {*}) are uniform, and the two 
sets ^^{■^pa{^J■a{<S) \ {*}) \ {*}) and 7/'f,(^h(iS) \ {*}) \ {*} are equal (see Definition all paths are equally 
likely. Consequently, no additional codeword bits can be determined. Thus, the extrinsic probability distributions 
for systematic bits in positions in ■i/ji,{fj.i,{S) \ {*}) \ {*} are uniform. One iteration of the basic turbo decoding 
algorithm has been performed and no additional codeword bits have been determined. Since there is no new 
information available to the first constituent decoder, no additional bits will be determined in the next round of 
turbo decoding either It follows that the decoder cannot determine the bits corresponding to the positions in the 
unique maximum-size turbo stopping set which is also a subset of £. Note that there is a unique maximum-size 
turbo stopping set which is also a subset of £, since the union of two turbo stopping sets is also a turbo stopping 
set. Conversely, if the decoder terminates at a set S, then there will exist subcodes Ca C Ca and Cb C Cf, of 
dimension > with support sets HaiS) \ {*} and /ih(5) \ {*}, respectively. Since the turbo decoder terminates, 
the two sets TT{tpa{i^a{<S) \ {*}) \ {*}) and ^^{^^{S) \ {*}) \ {*} are equal. From Definition^ it follows that S is 
a turbo stopping set and, since no erased bit-positions contained in a turbo stopping set can be determined by the 
turbo decoder, it must be the maximum-size turbo stopping set which is also a subset of £. ■ 

A. A (155, 64, 18) Turbo Code 

In [30], a particularly nice (3, 5)-regular LDPC code of length 155, dimension 64, and minimum Hamming 
distance 20 was constructed. The underlying Tanner graph has girth 8 which makes the code an excellent candidate 
for iterative decoding. This is the reason behind the selected code parameters. 

The turbo code is obtained by puncturing of a nominal rate- 1/3 turbo code with nominal rate- 1/2, constraint 
length V ~ A constituent codes defined by the parity-check matrix H(Z)) = {1 + D + D"^ + 1 + + D^). The 
last polynomial which is irreducible and primitive has been chosen as the parity polynomial making the constituent 
encoders recursive. The information block size is 64, the interleaver length is 72 due to dual termination [23], and 
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the interleaver is a dithered relative prime (DRP) interleaver [10], [31]. The puncturing pattern is designed using the 
algorithm in [32]. The minimum distance of 18 of the code has been computed using the algorithm in [7]. For this 
code there exists a turbo stopping set of size 17. The turbo stopping set is depicted in Fig. |2l In Fig.|2l the three 
upper rows of nodes correspond to the first constituent encoder, while the three remaining rows of nodes correspond 
to the second constituent encoder The nodes in row number 5(i — 1) + 1 give the bit-position in the turbo codeword 
of the corresponding systematic (i = 1) and parity {i = 2) bits. The nodes in row number 3(i — 1) + 2 and i + 2 
correspond to parity and systematic bits, respectively, from constituent encoder i, i ^ 1,2. The blue and dark green 
nodes correspond to systematic bits. The blue nodes are information bits (can be assigned freely), while the dark 
green nodes are redundant systematic bits (can not be assigned freely). The red and yellow nodes correspond to 
erased bits and punctured (parity) bits, respectively. The light green nodes correspond to parity bits. The arrows in 
between the upper three rows and the remaining three rows correspond to interleaving of erased information bits. 
The remaining part of the interleaver is of no concern for the following discussion. In fact, possible choices for Ca 
and Cb are the linear subcodes spanned by 

41) = (Oo, . . . , 00, lo, 10, 00, 11, 01, lo, 00, ... , 00, 00, Oo, Oo, 00, 00, Oo, . . . , Oo) 

= (Oo, . . . , 00, Oo, 00, 00, 00, 00, Oo, 00, ... , 00, 11, Oo, Oo, 10, 11, Oo, . . . , Oo) (3) 

and 

c[^' = (Oo, . . . , 00, lo, 01, Oo, Oo, 10, 01, Oo, Oo, 00, 00, 00, Oo, 00, 00, 01, 01, 00, 01, lo, Oo, . . . , 00, Oo, Oo, Oo, 
Oo, Oo, 00, ... , Oo) 

cf (Oo, . . . , 00, Oo, 00, Oo, lo, 01, 01, Oo, Oo, 00, 00, 00, Oo, 00, 00, 01, 01, 00, 01, lo, Oo, . . . , 00, Oo, Oo, Oo, 
Oo, Oo, 00, ... , Oo) 

cf ^ = (Oo, . . . , 00, Oo, 00, Oo, Oo, 00, 00, Oo, Oo, 00, 00, 00, Oo, 00, 00, 00, 00, 00, 00, Oo, Oo, . . . , 00, lo, Oo, Oo, 

lo,lo,00,...,Oo), (4) 

respectively. The symbol o indicates that the bit-position has been punctured. From Fig.|2]we get 

S = {10, 11, 15, 16, 18, 19, 69, 70, 73, 75, 76, 115, 116, 117, 123, 124, 126} 
^laiS) ^ {10, 11, 15, 16, 18, 19, 69, 70, 73, 75, 76, *} 
Aih(5) = {9, 11, 13, 14, 15, 17, 32, 34, 38, 39, 101, 104, 105, *} 
Va(/ia(5) \ {*}) = {8, 9, 11, 13, 43, 46, 47, *} 

M^^biS) \ M) = {7, 10, 11, 25, 61, 64, 65, *}. (5) 

Furthermore, it holds that x{Ca) = /ia(5)\{*}, xiCb) = t^biS)\M, and ^(V'a(x(a))\{*}) = MxiCb))\M, 
which shows that 5 is a turbo stopping set. 
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Fig. 2. Turbo stopping set of size 17 for tlie example (155, 64, 18) turbo code. The direct sum of the two upper (lower) sets of binary vectors 
(extended to length I with zeros) is the systematic part of the subcode Ca (Ci,). 



The turbo stopping set depicted in Fig.|2]is given as an example to illustrate the concept of a turbo stopping set. 
Actually, it is possible to find an exhaustive list of all turbo stopping sets of size less than some threshold using a 
modification of the algorithm in [7]. The details of the algorithm are outlined in Section [V] For this code there are 
2 turbo stopping sets of size 16 and 13 of size 17. 

B. A (201,64,21) Turbo Code 

From Fig. |2]it is apparent that some of the punctured bits can be reinserted without increasing the size of the 
depicted turbo stopping set. In fact, 27 out of the 31 punctured parity bits from the first constituent code can be 
reinserted. For the second constituent code, 19 out of the 30 punctured parity bits can be reinserted. The minimum 
distance of 21 of the resulting turbo code has been determined by the algorithm in [7]. Note that this is a constructed 
example, since it is possible to design a puncturing pattern of weight 15 that will give a length 201 and dimension 
64 turbo code of minimum distance 25 using the same mother code and the algorithm in [32]. The minimum 
distance of the unpunctured mother code is 27. 

C. Remarks 

The code considered in Section Illl-AI is not a rare example in the sense of having turbo stopping sets of size 
smaller than the minimum distance. We have found several examples of excellent turbo codes with this property. 
For further examples see Section IV-EI 

IV. Improved Turbo Decoding 

In [4], Pishro-Nik and Fekri consider improved iterative BP decoding of LDPC codes on the BEC. When standard 
iterative BP decoding fails, the improved algorithm chooses one of the unknown variables nodes and guesses its 
value. The decoding continues until either the transmitted codeword is recovered, the decoder does not progress 
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further, or the decoder reaches an inconsistency. The decoder is said to reach an inconsistency if all the variables 
nodes connected to a check node are known, but the check node is not satisfied. If the decoder has reached an 
inconsistency, then the value of the guessed variable node is changed and the decoding is repeated. This time 
the decoder will not reach an inconsistency, but the decoding can stop again. More sophisticated algorithms for 
improved iterative BP decoding of LDPC codes on the BEC were recently proposed by Ravisankar and Fekri 
in [2]. These algorithms improve upon the algorithms in [4]. Furthermore, the algorithms in [2] use the Tanner 
graph representation of the LDPC code actively to identify equivalent bit-positions when iterative BP decoding has 
stopped in a stopping set. In this context, two bit-positions are equivalent if and only if the knowledge of one of the 
bits implies the knowledge of the other bit after a series of message-passing in the subgraph of the Tanner graph 
composed of all unknown variable nodes and the neighboring check nodes of degree 2. 

Improved turbo decoding, the pseudo-code of which is given below, has the same structure as the algorithm 
in [4]. In more detail, it is based on guessing unknown systematic bit-positions and then decode until either the 
transmitted codeword is recovered, the decoder does not progress further, or the decoder reaches some kind of 
inconsistency. In the context of turbo decoding, however, the unknown bit-position to guess can be chosen more 
efficiently than in the context of improved iterative BP decoding of LDPC codes, since the forward and backward 
passes on the constituent trellises give information on legal paths. Bit-position selection is considered in more detail 
in Section ITV^Al 

In the pseudo-code below 7^''^, x — a,b, denotes the number of vertices of T^j^f^ at trellis depth j that are legal 
after the hh iteration, 
step 1 (Initialization): 

1) Choose the maximal number of iterations Zmax- 

2) Initialize J and T with zero. 

step 2 (Original turbo decoding; bit-position selection): 

1) Run turbo decoding as described in Section ITl-CI with a maximum of /max iterations, until either the codeword 
is recovered, or the decoder fails to progress further Let Ij < /,„ax denote the number of iterations carried 
out and increment T with Ij. 

2) If T = /max, or the codeword is recovered, make a decision based on c, and terminate the algorithm. 

(T) (T) 

3) Compute 7^0, ■•■'Ti/ for x = a,b and select a bit-position vt to guess based on these values. See 
Section HV- A! below for details. 

4) Initialize a list C with the ordered sets {{vt, 0)} and {{vt, 1)}. 

5) Initialize AAq and A4i with c. 

step 3 (Performing additional iterations; bit-position selection): 

1) Increment J. 

2) L = . . . , is chosen and removed from C. 

3) Initialize c with A^,(o) and set q equal to il"''' where '0a(/^a(O) — -^i"'' ™d j = \L\ — 1. 
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4) Run turbo decoding as described in Section lTl-Cl with a maximum of Imax — T iterations, but without assigning 
values to c in the initialization step of the algorithm, until the decoder fails to progress further, or there is a 
trelUs depth j in which all vertices in T^j^f^ have false forward or backward state metrics for some x, x ^ a, b. 
In this case the decoder is said to have reached an inconsistency. Let Ij < Z^ax — T denote the number of 
iterations carried out and increment T with /,/. 

5) If T = /max, or the decoder has not reached an inconsistency and Cj ^ * for all j such that fJ,a{j) ^ * and 
^aip-aii)) 7^ j = 0, . . . , — 1, make a decision based on c, and terminate the algorithm. 

6) Delete M^m r(i^i-i)- 

7) If the decoder has not reached an inconsistency, perform the following. 

a) Compute 7^^\ . . . , 7^^'' for .t = a, 6 and select a bit-position vt to guess based on these values. See 
Section HV- Al below for details. 

b) Add the two elements Lq = L U {{vt,0)} and Li = L IJ {(wt,!)} to the list C, and initialize 
M (0) (lioi-i) and X (0) with c. 

-^0,2'---J-^0,2 -^l,2"--'-^l,2 

step 4 (Repeating): Repeat step 3. 

The list C in improved turbo decoding can be implemented as a last-in first-out queue or as a first-in first- 
out queue. The first-in first-out implementation requires more memory than the last-in first-out implementation. 
The efficiency of an improved decoding algorithm with the above structure is very dependent on the selection of 
bit-positions to guess. 

A. Bit-Position Selection for Improved Turbo Decoding 

When turbo decoding fails to progress further, the unknown bit-positions constitute a turbo stopping set. Thus, 
guessing a bit-position in x{Cx) will free at least one additional bit. Some of the positions in xi^x) can be 
determined from the numbers 7^'^ of legal vertices at trellis depth j in TTj^j^ {I is the iteration number). In particular, 
the j\h systematic bit is unknown if 7^'^ = 1 and 7a'j+i = 2, or 7^ j+i = 1 and 7^'^ = 2. When selecting a 
bit-position to guess we would also like to free as many unknown positions as possible. We propose the following 
bit-position selection algorithm. 

1) For 2; = a, & do the following. 

[T) (T) 

a) Let Ix be the number of indices j with the property that 7^ j = 1 and 7^ j^-^ = 2. 

b) Let Wxj- be the largest non-negative integer such that there exists an index fx with the property that 

-y'^^ - 1 and -7*^' - • • • - -7^^' - 2 

^xj^-i - ^ ana -/^j^ - - l^.j^+w^j - ^■ 

c) Let Wx.r be the largest non-negative integer such that there exists an index rx with the property that 

-y'^) - 1 and -7*^^ - ... - ^'^^^ - 2 

d) h&i Wx = Taa3^{■WxJ■,Wx,r)■ 
2) If la > lb, or la = h and Wa > Wb, then set x — a. Otherwise, set x = b. 
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3) If Wx.f > w^-^r, then vt is set equal to fx — I if x = a and tt ^(/^ — 1) if x = 6. Otherwise, vt is set equal 
to if a; = a and ■JT~^{rx) ii x = b. 

B. Remarks 

We remark that several variations of the above bit-position selection algorithm are possible. In particular, one 
could use both the number of legal edges in each trellis section, or equivalently the edge entropy, and the number 
of legal vertices for each trellis depth, or equivalently the vertex entropy, in combination with the interleaver to 
improve the algorithm as described below. 

Let Vt denote a chosen systematic bit-position within a vertex entropy transition from level j at time t to level 
j + 1 at time t + 1 (a forward transition from level j at time t), or from level j at time t to level j + \ at time t—1 
(a backward transition from level j at time t) for one of the constituent trellises. The effective length of vt is the 
number of undetermined systematic bit-positions that will be determined by the forward-backward algorithm on the 
considered constituent trellis before any interleaving, if vt is guessed. To simplify notation we assume that vt is 
chosen based on the first constituent trellis. The effective length of vt is a positive integer smaller than or equal 
to the number of undetermined systematic bit-positions within the range [t, t + — 1] for a forward transition, or 
within the range [t — Ws,t—l\ for a backward transition. The positive integer Ws is the smallest integer such that the 
vertex entropy at time t + Wg, for a forward transition, or at time t — Wg, for a backward transition, is different from 
j + 1. A general upper bound on the effective length of vt is Ws- When j = 0, we can use the edge entropy to find 
the exact value of the effective length of vt- Let denote the smallest positive integer such that the edge entropy 
for the {t + We)th trellis section, i.e., for the transition from time t + w^ to time t + We + 1, for a forward transition, 
or for the — 1 — u'e)th trellis section, for a backward transition, is 2. A general upper bound on the effective 
length of Vt in this case is mm{ws,We)- The exact value is the number of undetermined systematic bit-positions 
within the range [t, t + mm{ws,We) — 1], for a forward transition, or within the range [t — mm{ws, We), t — 1], for 
a backward transition. The selection of bit-positions can be improved even further by actively using the interleaver. 
If we choose a bit-position vt with the property that both vt and ^{vt) are within vertex entropy transitions, then 
the performance will be improved. 

Finally, we remark that the simple version described in Section IIV-AI provides good results as indicated in 
Section HV^ below. 

C. Some Properties of Improved Turbo Decoding 

In this subsection we estabhsh some basic results of improved turbo decoding as described above. The following 
lemma is simple, but important, since the bit-position selection algorithm in Section HV-AI is based on this result. 

Lemma 4: Apply improved turbo decoding as described above using the bit-position selection algorithm in 
Section HV-AI Then, the channel value corresponding to the selected bit-position vt is an erasure. 

Proof: The bit-position selection algorithm in Section IIV-AI selects only (systematic) bit-positions vt S 

!T\ (T\ 

{0, 1} with the property that Tx,-k^{vt) 'x,L{vt)+i for x = a or 6, where iTxivT) = wt for x = a 
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and TTxivr) — t^{vt) for x = b. If the channel value is not erased, then we know the w^th information bit 
with probability 1. Since the w^th information bit is known with probability 1, and both constituent trellises are 
information bit-oriented, there is only a single legal edge out of each legal vertex at trellis depth TTxivr) for x — a 
or b. Consequently, the number of legal trellis vertices at trellis depth tTx{vt) + 1 is equal to the number of legal 
trellis vertices at trellis depth tTx{vt), and the result follows by contradiction. ■ 
Lemma 5: Let C denote a given PCCC with interleaver tt. Let S = S{Tr) denote a turbo stopping set, and erase 
all bit-positions in S. Then, choose any bit-position j G S, and do the following. 

1) Fix the bit- value in bit-position j to and perform turbo decoding until either the decoder fails to progress 
further, or the decoder reaches an inconsistency. If the decoder does not reach an inconsistency, denote the 
set of erased positions that remain when the decoder stops by Sj^\ 

2) Fix the bit-value in bit-position j to 1 and perform turbo decoding until either the decoder fails to progress 
further, or the decoder reaches an inconsistency. If the decoder does not reach an inconsistency, denote the 
set of erased positions that remain when the decoder stops by Sj^\ 

If the decoder does not reach an inconsistency in either of the two cases above, then the two sets iSj"'' and Sj^^ 
are equal. 

Proof: Fix the bit- value in bit-position j to c where c = or L Then the number of possible paths in the first 
constituent code is immediately reduced by a factor of 2, since the subcode Ca corresponding to S is linear. Let 

(c) 

this reduced set of legal paths in the first constituent code be denoted by P^j- Furthermore, the forward-backward 
algorithm for the first constituent code will determine additional bit-positions (which are previously unknown) 

(c) (c) (c) 

contained within a set 5^ j. For a given bit-position i G 5'aj' paths in ^ will have the same bit- value of c 
(depending on the value of c) in this bit-position. Since the subcode Ca is linear, all paths in P^'^j, where c denotes 
the complement of c, will also have a fixed bit-value of c in bit-position i. Thus, it holds that i from which 

(c) (c) 

it follows that j = since c and i both are arbitrarily chosen. For the second constituent code, several bit- 
positions are fixed due to extrinsic information from the first constituent code. However, we can apply the same type 
of arguments as above to show that the sequence of additional bit-positions determined by the forward-backward 
algorithm in the second constituent code is independent of c. The result follows by applying these arguments in an 
iterative fashion until there is no further progress. ■ 
Lemma 6: Apply improved turbo decoding as described above with the bit-position selection algorithm in 
Section HV-AI For any two elements 

L = . . . , (Lf I-))} G £ and Z = {(L^ , ), . . . , (Zf '"^ Zf l^^))} G C 

with the property that \L\ = |Z|, it holds that l'^^'' = for all i, i = 0, . . . , \L\ — 1, i.e., the actual bit-values 
in the guessed bit-positions do not influence on which bit-positions are selected next by the bit-position selection 
algorithm in Section HV-AI as long as no inconsistency is reached. 

Proof: The result follows directly from Lemma [S] ■ 
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We remark that due to Lemma |6l we can reduce the number of times we need to run the bit-position selection 
algorithm from Section ll V- Al when performing improved turbo decoding. 

Theorem 2: Improved turbo decoding is ML decoding on the BEC when Zmax oo. 
Proof: The proof is two-fold. First we prove that if the algorithm terminates, then we have an ML decoder. 
Secondly, we prove that the algorithm will always terminate. 

1) It follows from the pseudo-code above (step 2, item 2), and step 3, item 5)) that if the algorithm terminates, 
then the decoder has not reached an inconsistency and Cj ^ * for all j such that /iaO) 7^ * and ipa {lJ-a{j)) 7^ *, 
j = 0, . . . , N — 1, since Z^ax ^ 00. The original turbo decoding algorithm will not introduce bit errors and 
neither will the improved turbo decoding algorithm due to items 5) and 7) in step 3. In more detail, the 
algorithm will not terminate if the decoder has reached an inconsistency, and no further guessing is performed 
if this is the case. Thus, if the improved turbo decoding algorithm terminates, then transmitted codeword is 
recovered, or there exists a different turbo codeword c' with support set x(c') ^ £ where £ denotes the subset 
of remaining erased bit-positions. In the latter case the transmitted codeword is not recovered. An ML decoder 
will not be able to determine the transmitted codeword in the latter case either, since both codewords c and 
c + c' where c denotes the transmitted codeword are equally likely to have been transmitted. Thus, improved 
turbo decoding is ML decoding. 

2) As the algorithm proceeds, erased bit-positions are guessed. Each time an element L is removed from the list 
C, turbo decoding is performed. If the decoder does not reach an inconsistency and cj = * for some j such 
that /ia(j) 7^ * and ipa{l^a{j)) 7^ *, J = 0, . . . , iV — 1, then a new erased bit-position is guessed (see the 
bit-position selection algorithm in Section ITV- Al and Lemmal^for details). The decoder will always terminate, 
since there is a finite number of bit-positions to guess. 

■ 

D. Numerical Results 

Here we present some simulation results of improved turbo decoding on the BEC. The simulated frame error 
rate (FER) is presented in Fig. |3] for the (155,64,18) turbo code introduced in Section UlI-AI We have used the 
bit-position selection algorithm described in Section HV-AI in the simulations. The truncated union bound (TUB) in 
Fig.|3]is computed from the first 5 non-zero terms of the code's weight distribution. The near ML decoding curve 
is obtained using improved turbo decoding with a large number for /,nax- In Table |l] we have tabulated, for different 
values of the channel erasure probability e, the empirical value of Imax such that improved turbo decoding is near 
ML decoding. In this context, improved turbo decoding is near ML decoding when the fraction of ML-decodable 
frame errors observed in the simulation is ^ 0.05. The corresponding estimated values of the expected number of 
iterations E\T] are tabulated in the third row of the table. The gap between the TUB and the near ML performance 
curve at moderate-to-high values of e is due to the fact that only a limited number of codewords are taken into 
account in the summation of the union bound. The two remaining curves show the FER for two different values 
of /max- Observe that when /max is increased, the performance improves. In Table Hll estimated values of E\T] 



Febraaiy I, 2008 



DRAFT 



SUBMITTED TO IEEE TRANSACTIONS ON INFORMATION THEORY 



16 



of improved turbo decoding are tabulated for different values of e and Imax- From Table |II| we observe that the 
difference in the estimated values of E[T] for Zmax = 170 and Imax = 10 decreases when e decreases. In particular, 
for e ~ 0.40, there is almost no difference in the expected number of iterations. However, as can be seen from Fig.|3l 
there is a large difference in performance. The numbers in Tables HI and Hll are based on more than 1000 observed 
frame errors for e = 0.40, 0.45, 0.50, 0.55, and more than 100 frame errors for e = 0.35. Similar performance 
improvements have been observed for the (3600, 1194,49) turbo code from [10], 




Fig. 3. FER on the BEC of the (155, 64, 18) turbo code from Sectioning 



V. Turbo Stopping Set Enumeration 
A. Convolutional Codes and Trellises 

An edge-labeled directed graph is a triple [V^E^A), consisting of a set V of vertices, a finite set A called the 
alphabet, and a set E of ordered triples (u, a, v'), with v,v' and a e A called edges. The edge (w, a, v') begins 
at V, ends at v' , and has label a. 

Let C denote a linear [n, fc, v) convolutional code over some finite field Fq of q elements, where v is the 
constraint length or the code degree. In this work the convolutional code symbols are taken from the binary field 
F2 = GF{2). A convolutional code can be defined by an {n — k) x n polynomial parity-check matrix Ti{D). We 
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TABLE I 

Estimated expected number of iterations E[T] and empirical Zmax such that improved turbo decoding is near ML 

DECODING FOR THE (155, 64, 18) TURBO CODE FROM SECTIOn HII-AI 



e 


0.55 


0.50 


0.45 


0.40 


0.35 


^max 


2600 


2400 


2000 


1100 


400° 


EIT] 


402 


49 


4.4 


1.46 


1.10 



"The estimate is less reliable, since only 6 ML-decodable frame en'ors have been observed compared to about 50 for e > 0.35. 

TABLE II 

Estimated expected number of iterations E[T] of improved turbo decoding for different values of e and Zmax for the 

(155, 64, 18) turbo code from Section I!iI-AI 





0.55 


0.50 


0.45 


0.40 


0.35 


10 


8.3 


5.1 


2.42 


1.42 


1.10 


170 


74 


19.5 


3.57 


1.45 


1.10 



assume in general a canonical parity-check matrix [33]. The maximum degree of the polynomials in the ith row is 
the ith row degree, denoted by Vi. 

Let be the matrix H(0), and let be the matrix diag(D^i, . . . , D''"-'')li{D-^) with D = 0. When a 
matrix is given an integer interval subscript we mean the submatrix consisting of the columns with indices in the 
interval. The columns in a matrix are indexed from left to right with positive integers. 

7 ) Trellis Representation of Convolutional Codes: The minimal trellis of C can be constructed from a parity- 
check matrix of the code as outUned in [34]. The minimal trellis can be regarded (after an initial transient) as the 
infinite composition of a basic building block which is called the trellis module. The trellis module T = {V, E, Fq) 
of C is an edge-labeled directed graph with the property that the vertex set V can be partitioned as 

T/ = U ^1 U • • ■ U K (6) 

such that every edge in E begins at a vertex in Vi and ends at a vertex in V^+i, for some z, i = 0, . . . , rt — 1. The 
depth of the trellis module is n. The ordered index set X = {0, 1, ... , n} induced by the partition in Q is called 
the time axis for T. The partition in (|6j also induces a partition E ^ EqU EiU ■ ■ ■ UE„-i of the edge set E where 
Ei is the subset of edges that begin at a vertex in Vi and end at a vertex in V^+i. 

Define bo = and bi ~ rank[H^_j^]^ „], i = 1, . . . , n, and /o = and = rank[H{^j], i = 1, . . . , n. The vertex 
set Vi is a vector space of dimension clim(\/i) = u — n + k + fi + bn-i < ly + n — k, from which it follows that 
dim(Vb) = dim(V^„) = The edge set Ei is also a vector space of dimension dim{Ei) = L' — n+k+ fi + bn-i-i + 1- 

Let Zinfo be the subset of Z\ {n} consisting of all integers i with the property that bn-i = fcn-i-i- Furthermore, 
we assume without loss of generality that 5„ = which implies that G 2info- Let rii = j + I where j is 
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the largest non-negative integer < n — i — 1 such that bn-i-i 7^ ■ • ■ 7^ hn-i-j-i for every i £ Tmio- As argued, 
for instance in [26], there are n — k time instances z e X \ Xinfo in which there is only a single edge out of each 
vertex in Vi. By sectionalization the depth of the trellis module T can be reduced to k. This reduced trellis module 
is called an information bit-oriented trellis module and is denoted by Tinfo = (Vinfo, £'info, UigXinfo^fJ*') where 
^nfo = Uigi.^j^u{n}^ and i^info = ^ieiit.ta-^i- The edge set E'^ is the set of paths that begin at a vertex in Vi and 
end at a vertex in Vi+m ■ The label of an edge in E[ is the label sequence along the defining path which is a g-ary 
sequence of length n^. The edge set E[ is a vector space of the same dimension as Ei. 

In the trellis module Ti„fo there are q edges out of each vertex in Vinfo \ Vn- Thus, we can assign a g-ary input 
label to each edge e G £^info, and the trellis module Tinfo can used for encoding. 

2) Trellis Representation of Subcodes of Convolutional Codes: A trellis representing subcodes of C can be 
written (after an initial transient) as the infinite composition of an extended trellis module T ~ {V,E,F2) where 
V — VqU ■ ■ - UVn and _E = i?o U • • • U En-i are partitions of V and E, respectively. Each vertex of Vt corresponds 
to a subspace of Vi, and each edge of Ei corresponds to a subspace of Ei. The number of distinct fc-dimensional 
subspaces of an n-dimensional vector space over Eg, k = 1, . . . ,n, denoted by S{k, n, q), is [35, p. 444] 



_ 1 \ . . . (r,n-k+l 

S{k,n,q) = 



(g" - l)(g""^ - 1) • • • (g- - ' ^ - ij _ rn 

k 



(g^--l)(g'^-i-l)...(g-l) 
from which it follows that 

dim(Vi) ^-,^^1 dim(_Ei; 

and \Ei\ = l+ ^ 



dim(l^O 
j 



dim(£'i 
J 



q j=i 

The left (resp. right) vertex of an edge e is denoted by f^(e) (resp. i;^(e)). The label of an edge e is denoted 
by c(e). Note that the edge e could either be an edge in the trellis module T or in the extended trellis module T. 

The connections in the extended trellis module T are established as follows. An edge e G Ei corresponds to 
a subspace of Ei of dimension d{e) and basis {eo(e), . . . ,ed{e)-i{e)}- The vertex in Vi that corresponds to the 
vector space spanned by {u^(eo(e)), . . . , w^(e(j(g)_i(e))} is connected to the vertex in V+i that corresponds to 
the vector space spanned by {w^(eo(e)), . . . , z;^(erf(g)_i(e))} by the edge e. The binary label c(e) of e is 1 if at 
least one of the g-ary labels c(ej(e)) of ej(e) G Ei, j ~ Q, . . . , d{e) — 1, is non-zero. Otherwise, it is 0. In the case 
of parallel edges in T with the same label, we keep only one. 

Note that an information bit-oriented extended trellis module Tjnfo can be obtained by sectionalization as described 
above. A trellis T[nfo constructed as the infinite composition of the trellis module Tlnfo has paths that are in one- 
to-many correspondence with subcodes of C. The label sequence of a path in T[nfo is a binary sequence where the 
set of 1-positions is equal to the support set of the subcodes represented by the given path. Since distinct subcodes 
could have equal support sets, there could be paths in T[nfo that have equal label sequences. 

Input labels can be assigned to the edges in the trellis module Tlnfo, and thus Tinfo can be used for encoding, but 
the encoding is yiot one-to-one, since there could be more than one path in %nio with the same input (and output) 
label sequence. 
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TABLE in 

Vertex and edge complexities of the trellis modules Tinfo and finfo for (n, 1, u) binary convolutional codes 



V 


2 


3 


4 


5 


6 




4 


8 


16 


32 


64 




8 


16 


32 


64 


128 


M{2lnfo) 


5 


16 


67 


374 


2825 


'/'('finfo) 


16 


67 


374 


2825 


29212 



Example 2: Consider the (2, 1,2) binary convolutional code defined by the parity-check matrix H(L)) = (1 + 
1 + D + D^). The trellis module Tinfo and the extended trellis module Tinfo are both depicted in Fig.|4] Note 
that the trellis module Tinfo is non-linear. 




(a) (b) 



Fig. 4. (a) Basic trellis module Tinfo, and (b) extended trellis module Tinfo of the convolutional code from Example I2I 



The numbers of vertices and edges of an (extended) information bit-oriented trellis module Tnfo divided by k 
are called the vertex and edge complexities of Tnfo, and are given, respectively, by 

M(T^info) = ^ l^d and 0(Ti„fo) = i ^ \E,\. 

The vertex and edge complexities of Tnfo and Tnfo are tabulated in Table IIIII for different values of v for (n, 1, v) 
binary convolutional codes. 

B. The Uniform Interleaver 

Consider an (n, fc, v) systematic convolutional code which is terminated to the all-zero vertex at trellis depth 
I > \v/k'\k in Tinfo where / is assumed to be a multiple of k. The resulting linear block code C has length 
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S = {I/k)n and dimension I — v. When v = Q, the convolutional code is actually a linear block code in which 
case we choose I ^ k. 

Partition all the subcodes of C of dimension d, d ~ 1, . . . , I — ly, into equivalence classes based on their support 
sets. In particular, all subcodes within a specific subcode class are required to have the same support set, but the 
subcodes may have different dimensions. 

We define the subcode input-redundancy support size enumerating function (SIRSEF) of C. The SIRSEF has the 
form 

I 5-1 

A^(H^,Z) = ^^aS,,M^'"^^ (7) 



w=l z=0 

where ^ is the number of subcode classes of C of input support set size w and parity support set size z. When 
analyzing the performance it is useful to group the terms in the SIRSEF according to input support set size. The 
conditional SIRSEF 

5-7 

AZ{Z) = Y.''Z,zZ' (8) 

enumerates subcode classes of different parity support set sizes associated with a particular input support set size. 
The conditional SIRSEF A^{Z) and the SIRSEF A'^ {W, Z) are related to one another through the following pair 
of expressions 

A-iW. Z)^± W-AZ{Z) and A^iZ) = -1 ■ '-^^^^ \ (9) 

w — l 

Let yl^^(Z), X = a,b, denote the conditional SIRSEF for the constituent code Cx of a given PCCC C with 
interleaver length /. We assume that the interleaver is uniform. With a uniform interleaver, the (^) possible sequences 
of length / and weight w occur with equal probability at the output of the interleaver when the interleaver is fed with 
a length-/ and weight-u; sequence. Let S^{W, Z) denote the input-redundancy turbo stopping set size enumerating 
function (IRTSSEF) of C. The IRTSSEF has the form 

I 2(5-7) 

s''iw,z) = Y, E 4.^"^' (10) 

w=l z=0 



where s^^ ^ is the number of turbo stopping sets of C of input size w and parity size z. The conditional IRTSSEF 

2(5-7) 

S'JZ)= J2 <zZ' (11) 

z=0 

enumerates turbo stopping sets of different parity sizes associated with a particular input size. The conditional 
IRTSSEF S^{Z) and the IRTSSEF S'^{W, Z) are related to one another through the following pair of expressions 

S^iW, Z) ^ ± W-SiiZ) and Si^Z) = -1 • ^^^^^ | (12) 

w — l 

A PCCC with a uniform interleaver has a uniform probability of matching a given support set in Aj^" (Z) with any 
given support set in A'^''{Z). Thus, it follows that the conditional IRTSSEF for C is 

\w} 
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The turbo stopping set size enumerating function (TSSEF) for C is 

2S-I min(i,/) 
i—l w—1 

and sf is the number of turbo stopping sets of size i. If only 1 -dimensional subcodes are considered in the constituent 
conditional SIRSEFs and A'^^''{Z), then we get the weight enumerating function (WEF) for C. 

1) The (7,4) Hamming Code: We consider the (7,4) Hamming code in its cyclic form in which case jy = 0, 
I = k — 4, and S = n ^ 7. The SIRSEF of the Hamming code is 

W{3Z^ + Z^) + W'^{3Z + 3Z'^ + 6Z^) + W^il + 3Z + 12Z^ + 4Z^) + W*{3Z + 3Z^ + Z^). 

The IRTSSEF for the PCCC C with a uniform interleaver is 

74 I 1 c '75 I n OK '76\ 1 TJ/2/1 r rT2 , o ryZ . ry r '74 i Ci '75 I G '76\ i 



(15) 



^^^(2.25^* + 1.5Z^ + 0.25^") + W^X.hZ"- + 3Z'' + 7.5Z* + + 

^^^^(0.25 + \.hZ + 8.25^2 + 20Z^ + 42Z^ + 24Z^ + 4Z^) + VF'^(9Z2 + + ISZ"* + + Z*^) 
from which the TSSEF can be calculated. The result is 

0.25X3 + 3X^ + \3.hX^ + 38X6 _^ 66.25X^ + 45X^ + lOX^ + X^". 

Note that the WEF for C is 

1 + 0.25^3 + 3X^ + 7.5^5 + 3X^ + 0.25X^ + X^". 

We can check the result in ( I15> by computing the IRTSSEFs of the PCCCs constructed using all the 4 ! = 24 
possible interleavers. The results are tabulated in Table IIVI Only two types of IRTSSEF are possible. It is easy 
to verify that the average over all possible interleavers is equal to the expression in ( I15> . Note that the WEF is 
dependent on the interleaver while the non-codeword TSSEF is the same for all interleavers. 

2) Convolutional Codes as Constituent Codes: Let T'^== ( W, Z, F, S), x = a,b, enumerate all subcode classes 
of Cx constructed from trellis paths in constituent trellis TTj^j^ leaving the all-zero vertex at trellis depth zero, and 
remerging into the all-zero vertex at or before trellis depth /, with possible remerging into the all-zero vertex at 
other depths in between, subject to the constraint that, after remerging, the paths leave the all-zero vertex at the 
same trellis depth. In general, 

/ s-i I L7/2J 

T<^- (w, z,t,j:)^Y.T.T.11 t^^^^aW-z^vj:^ (i6) 

w—1 z—0 J— 2 a—1 

where t^^^ ^ a number of subcode classes of of input support set size w and parity support set size z 

constructed from trelUs paths of length 7, and with a remergings with the all-zero vertex. Notice that each subcode 
class in T'^^ (VF, Z, F, S) of input support set size w and parity support set size z constructed from trellis paths 
of length 7, and with a remergings with the all-zero vertex, gives rise to (^^^'^) subcode classes with the same 
input and parity support set sizes. Thus, the conditional SIRSEF A^'' (Z) can be written as 



s-i 



z=0 



I L7/2J 



7^2 cr^l 



w.z.'y,a' 



(17) 
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TABLE rV 

IRTSSEFS. TSSEFS, AND WEFS OF THE PCCCS CONSTRUCTED USING ALL THE 4 ! = 24 POSSIBLE INTERLEAVERS AND WITH THE (7, 4) 



Hamming code as constituent codes . 



TT 


S"^(W,Z), S^{X), and WEF(X) 


3210 








3201 








3120 








3102 








3012 








3021 








2310 




= 


l + W{2Z* + 2Z'') 


2301 




+ 


W'^{Z^ + 4Z3 + 7^4 + 6^6) 


2130 




+ 


W^{2Z + 8Z2 + 20^3 + A2Z'^ + 24^^ + AZ'^) 


2031 




+ 


W*{9Z^ + 18Z« + 15^4 + 6Z5 + Z^) 


1230 






1 + SX" + 14X5 _,_ 3g;((-6 _,_ QQx'' + 45X* + 10X9 + X^" 


1320 


WEF(X) 




1 + SX" + 8X5 _,_ 3j(^6 _,_ xw 


1302 








1032 








0231 








0132 








0312 








0321 








2103 
2013 
1203 
1023 
0213 
0123 


+ 
+ 

S^(X) = 
WEF(X) = 


W^{1 + 9Z^ + 20Z^ + 42Z-^ + 24Z5 + 4Z6) 
Vy*(9Z2 + 182=* + ISZ"' + 6Z5 + Z^) 

l + X^ + SX-i + 12X5 _,_ 3gj(^6 _,_ Q-jx'^ + 45X8 + 10X» + X^" 
1 + X3 + SX" + 6X5 _,_ 3jc6 + X'' + X^" 



Finding a closed-form expression for the conditional SIRSEF (as we did in Section IV-B.ll when the constituent 
codes were Hamming codes) for a given (large) value of the interleaver length / is difficult. For this reason, we 
will use an algorithmic approach to compute the most significant terms of the conditional SIRSEF in Hit . One 
approach is to use the algorithm to be described in Section IV-CI with only one constituent code. See Section IV-CI 
below for details. 

C. Enumeration of Small-Size Turbo Stopping Sets for a Particular Interleaver 

Let Tj^fo and TJ^fo denote the information bit-oriented and the extended information bit-oriented trellis modules 
of constituent code Cx, x = a,b. We assume that T^f^ and T^^^ both have input and output labels assigned to the 
edges. The trellises constructed from the trelhs modules T^f^ and Tj^f^ are denoted by T^j^f^ and 7^^^, respectively. 

Let denote the subset of {0, l}^^' of label sequences of paths that begin and end at the all-zero vertex at 
treUis depths and / of the trelUs Tl^f^, x = a,b. Finally, let Sc — Sc{K,Sc\,Sc'^,Tr) denote the PCCC with 
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information length K, constituent encoders Sc^ and iScj, and interleaver tt. Note that the support sets of words 
of Sc are in one-to-one correspondence with all turbo stopping sets of C. Thus, the minimum Hamming weight of 
Sc is equal to the minimum turbo stopping set size of C. In general, the set of turbo stopping sets can be obtained 
by turbo encoding using the constituent encoders iSc„ and Sc^- Note that there could exist several paths in 7^^^^ 
with the same input label sequence from which it follows that the complexity of turbo encoding could be more 
than linear in K. 

Let n(iSc,T) be the problem of finding all words of Sq of Hamming weight < t. This problem is equivalent 
of finding an exhaustive list of all turbo stopping sets of size < r. To simplify notation we assume below that the 
I — K redundant systematic bits appear at the end of the input block. 

A constraint set F is a set {{pi,Up.) : Up- G {0,1} Vp; G Fp}, where Fp C {0, . . . , A' — 1} is a set of 
distinct positions. For any constraint set F, let U^^^ be the set of length-_ftr vectors {u = (uq, . . . , uk-i) '■ Uj = 
u if (j, u) G F, Uj G {0, 1} if j ^ Fp}. Let the length I = 1{F) be the number of constraints. We will start with a 
constraint set F of length I of the form {(0, uq), (1, ui), . . . ,{l — 1, u/^i)}, i.e., it applies consecutively to the first 
I positions. When the turbo interleaver tt acts on F, we obtain a new constraint set ttF = {(7r(pi), Up.)}, where in 
general the constrained positions are scattered over the input block. 

Let Sjf^ be the subset of Sc that is obtained by encoding the input vectors in C/(^\ let be the minimum 

Hamming weight of , and let w'{F) be any lower bound for w{F). The pseudo-code of the algorithm to solve 
n(5c, t) is given below. Note that the algorithm has the same structure as the algorithm proposed by Garello et al. 
in [8] to solve n(C, r). We will refer to this algorithm as the GPB algorithm. However, there are some differences 
that we will discuss below. 

/* Find all words of Sc of Hamming weight t */ 
Add an empty constraint set -F to a 
previously empty list L of constraint sets, 
(t) If L is empty, terminate the process. 
Otherwise, 

choose and take out a constraint set F from L. 
If w'{F) < T, then 

If the length / of -F is equal to K then: 

The single vector in U^^^ produces 

low-weight words in Sc which are saved. 
Otherwise, 

construct two new constraint sets: 

F' = FU{(/,0)} and F" = F U {{1,1)} . 
Add F' and F" to L. 
Proceed from (f ) . 

Let be the subset of words generated by the constituent encoder Sc^ when the input vectors are contained in 
[/(^), and let Wa{F) be the minimum Hamming weight of slj'\ Select any vector from C/'^' as an input sequence 
of Sca- After 1{F) time units the encoder has reached a subset {cr^ q , . . . , cr^ p(F)-iJ' cardinality p{F) of the set 
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of trellis vertices of TTJJj^ at trellis depth 1{F). The trelUs path of TT^j^ from the all-zero vertex at trellis depth to 

( F) ( F) ( F) 

vertex / at trellis depth 1{F) of minimum Hamming weight is denoted by c)j / . Let w{c)^ /) be the Hamming 

( F) ( F) ( F) 

weig ht of c^7, and let w{cr'^y ,1{F)) be the minimum Hamming weight of any path from vertex / at trellis 
depth 1(F) to the all-zero vertex at trelUs depth /. In general, it holds that 

Wa{F)= min (wicZ^)+wiai'^lliF))). (18) 

i=0,...,p(F) — 1 \ / 

The weights 'w{a^J^^ ,1{F)) can be computed in a preprocessing stage using the Viterbi algorithm. Actually, the 
weights w{<Tf^^ KP)) depend only on the vertex a\^^ if 1{F) is not too close to K and TT^Jj^ is non-catastrophic. 

( F) 

This reduces the memory requirements. However, the weights ^(c^ ^ ) have to be computed during the course of 

( F) 

the algorithm. In general, the weights w{c\ - ) can be computed by a constrained Viterbi algorithm. Note that in 
the original GPB algorithm there is no need to apply a constrained Viterbi algorithm here, since p{F) = 1, and 

(F) 

there is a unique path in T^J^j^ from the all-zero vertex at trellis depth to the vertex cr^ q at trellis depth 1{F). 

Similarly, let S^J^^ be the subset of words generated by the constituent encoder Sc^ when the input vectors are 
contained in U^'"^\ Also, let Wb^nF) be the minimum Hamming parity weight of S^J^\ We have 

t^bound 

(F) = Wa{F) + Wb{TiF) < w{F). (19) 

Note that Whound{F) is a lower bound on w{F), since the sequence of input bits giving the minimum-weight path 
in the second constituent encoder trellis T^f^j^ is not necessarily an interleaved version of the sequence of input 
bits giving the minimum- weight path in the first constituent encoder trellis "7^°^. The value of Wb{TTF) can be 
determined by the use of a constrained Viterbi algorithm. Since the positions of ttF are in general not consecutive, 
the complexity of calculating the value of WbinF) by a constrained Viterbi algorithm is larger than the complexity 
of calculating the weig hts w{c['^.l), i = 0, . . . , p{F) - 1, needed in ( fT8t . 

In [7] we outlined several improvements to the basic GPB algorithm for solving n(C, r). All of the improvements 
described in the context of solving n(C, r) can be applied when solving n(iSc, r). 

From Table Hn] we observe that the edge complexity of Tinio is large compared to the edge complexity of Tinio 
even for = 4. To reduce complexity we propose to remove some of the edges from TJ^jfo. For instance, all edges 
in Tj^jfo that correspond to edge subspaces of T^^^ of dimension > a, for some integer a > 2, can be removed. 

D. Remarks 

We remark that in principle every trellis-based turbo code weight spectrum computation or estimation algorithm 
can be adapted in a straightforward manner to find small-size turbo stopping sets. The only requirement is that 
the basic trellis module is substituted with the extended trellis module introduced above. We have considered the 
impulse methods by Berrou and Vaton [12], Vila-Casado and Garello [11], and Crozier et al. [10] with promising 
results. 
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E. Numerical Results 

We have applied the algorithm from Section IV-CI with the improvements from [7] on a few example codes. 
The example codes are constructed without considering turbo stopping sets. Consider the (828, 270, 36) turbo 
code constructed by Crozier et al. in [10]. The code has an optimized minimum distance, is based on a DRP 
interleaver, and is constructed from nominal rate- 1/2, v ~ ?> constituent codes defined by the parity-check matrix 
H(Z?) = (1 + D + 1 + + D^). The last polynomial which is irreducible and primitive has been chosen 
as the parity polynomial making the constituent encoders recursive. For this code there are 59 (0), 58 (58), and 
283 (283) stopping sets (codewords) of size (weight) 33, 36, and 37, respectively. The minimum turbo stopping set 
size is smaller than the code's minimum distance. This is typically what happens for both short and moderate-length 
distance-optimized DRP interleavers. With the uniform interleaver, however, there are 

0.08538 (0.08538), 1.245 • 10"^ (1.245 • 10^^), 0.01958 (0.01958), 0.66860 (0.66860), 1.91184 (1.90691), 

0.01171 (0.00598), 0.27896 (0.22497), 2.55047 (2.44926), 3.90238 (3.77258), 0.50298 (0.28068), 

2.48949 (1.83803), 7.22456 (6.12582), 8.24370 (6.66759), 7.72356 (4.50883) 
stopping sets (codewords) of size (weight) 9, 10, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, and 23, respectively. 
Thus, small-size turbo stopping sets is not a problem with the uniform interleaver for these parameters. As another 
example consider an interleaver length of 1200. In [10], Crozier et al. constructed an impressive DRP interleaver 
of length 1200 with a dither length of 8. The corresponding turbo code has length 3600, dimension 1194, and 
minimum distance 49. For this code we have found turbo stopping sets of size 47. Results for a range of interleaver 
lengths are given in Fig. [S] The turbo codes are constructed from the same nominal rate-1/2 constituent codes 
as the (828, 270, 36) code above. For each interleaver length between 32 and 320, in which the dither length is a 
divisor, we have found the best (in terms of turbo code minimum distance) DRP interleaver with a dither length of 
4. Also, for the same interleavers we have found the minimum turbo stopping set sizes. These results are plotted in 
Fig. |5l Note that for several interleaver lengths the minimum turbo stopping set size is smaller than the minimum 
distance. 

VI. Conclusion and Future Work 

In this work we have considered the finite-length analysis of turbo decoding on the BEC. In the same way as 
iterative BP decoding of LDPC codes is simpler on the BEC than on other channels, turbo decoding can also be 
simplified on this channel. Based on this simplified turbo decoding algorithm we have introduced turbo stopping 
sets by adapting the concept of stopping sets from the theory of iterative BP decoding of LDPC codes. These 
turbo stopping sets characterize turbo decoding on the BEC, and an exact condition for decoding failure has been 
established as follows. Apply turbo decoding until the transmitted codeword has been recovered, or the decoder 
fails to progress further. Then the set of erased positions that will remain when the decoder stops is equal to the 
unique maximum-size turbo stopping set which is also a subset of the set of erased positions. Furthermore, we have 
presented some improvements of the basic turbo decoding algorithm on the BEC. The proposed improved turbo 
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Fig. 5. Minimum distance and minimum turbo stopping set size as a function of interleaver length for distance-optimized DRP interleavers 
witli a ditlier lengtli of 4. 



decoding algorithm has substantially better error performance as illustrated by simulation examples. In the second 
part of the paper an expression for the turbo stopping set size enumerating function under the uniform interleaver 
assumption was derived. Also, an efficient enumeration algorithm of small-size turbo stopping sets for a particular 
interleaver was given. The solution is based on the algorithm proposed by Garello et al. in 2001 to compute an 
exhaustive list of all low-weight codewords in a turbo code. In fact, it turns out that every trellis-based weight 
spectrum computation or estimation algorithm for turbo codes can be adapted to the case of finding small-size turbo 
stopping sets. In particular, the impulse methods by Berrou and Vaton, Vila-Casado and Garello, and Crozier et al. 
can be adapted in a straightforward manner 

One interesting topic for future work is the design of interleavers in which one considers both low-weight 
codewords and small-size turbo stopping sets. Trellis-based interleaver design algorithms can in a similar manner 
be adapted to this problem using the extended trellis module. 

Finally, we remark that the findings in this paper can be adapted in a fairly straightforward manner to other 
turbo-like codes, e.g., RA codes, serial concatenated convolutional codes, and product codes. 
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